Supervised Multivariate Learning with Simultaneous Feature Auto-Grouping and Dimension Reduction

Abstract Modern high-dimensional methods often adopt the ‘bet on sparsity’ principle, while in supervised multivariate learning statisticians may face ‘dense’ problems with a large number of nonzero coefficients. This paper proposes novel clustered reduced-rank (CRL) framework that imposes two joint matrix regularizations to automatically group features constructing predictive factors. CRL is more interpretable than low-rank modelling and relaxes stringent sparsity assumption variable selection. In this paper, new information-theoretical limits are presented reveal intrinsic cost seeking for clusters, as well blessing from dimensionality learning. Moreover, an efficient optimization algorithm developed, which performs subspace clustering guaranteed convergence. The obtained fixed-point estimators, although not necessarily globally optimal, enjoy desired statistical accuracy beyond standard likelihood setup under some regularity conditions. kind information criterion, its scale-free form, proposed cluster rank selection, has rigorous theoretical support without assuming infinite sample size. Extensive simulations real-data experiments demonstrate interpretability method.